The present invention relates to computer vision systems. More particularly, the present invention relates to an improved apparatus and method for determining the characteristics of an articulated member.
Computer vision is a development in the broader field of computer systems. One of the goals of computer vision systems is the ability to recognize objects contained in electronic images. For example, a video camera may record a single image or a series of images and relay the information to a computer system. The computer system may then be able to determine the relative position of objects, persons, animals or any other images within the computer image.
Several types of applications exist for computer vision systems. By way of example, in industrial applications, a computer vision system may be utilized to monitor and control a robotic arm used in a fabrication system. Computer vision systems may also be used for pattern recognition purposes, manufacturing quality control, motion analysis and security.
FIG. 1 illustrates a typical scene 2 that a computer vision system may be used to analyze. Scene 2 may include a subject 4 and a background 5. Subject 4 can be any object which the computer vision system may be used to analyze. By way of example, in the illustrated example, subject 4 is a person. Background 5 may include any part of scene 2 that is not part of subject 4. In the illustrated example, background 5 includes the sun, clouds, the sky and hills.
FIG. 2 illustrates an electronic image 10 derived from scene 2 of FIG. 1. Electronic image 10 may include an electronic representation of the background 25, and a subject image 20 of subject 4. Typically, computer vision systems are concerned with various aspects of subject image 20. In the illustrated example, several features of subject image 20 are enumerated 20a through 20h. By way of example, the enumerated features may include a center of mass point 20a, a body box 20b, a head box 20c, an arm box 20d, a forearm box 20e, a hand point 20f, an elbow point 20g, shoulder box 20h and a shoulder point 20i. However, in general, any type of suitable characteristic of subject image 20 may be obtained through the computer vision system.
As will be discussed in more detail below, the prior art methods of determining the characteristics of subject image 20, are often times complex and unable to be accomplished in real time. For example, one method of subject characterization is through the use of stereo vision. In stereo vision, two cameras are used to capture the electronic images of a subject, e.g., subject 4 of FIG. 1. One advantage of the stereo vision method is that a 2-D or a 3-D image may be generated.
However, problems exist with registration in a system which uses stereo vision. That is, it is often difficult to match the two images captured by the separate video cameras to create a coherent single image. The registration problem typically leads to faulty characterization of subject image 20. Additionally, the use of two cameras add to the expense and complexity of the computer vision system. Also, the stereo vision system is typically not capable of characterizing subject image 20 in real time.
Another method of subject characterization is through the use of markers. Typically, markers are placed at various points on a subject that the computer vision system would like to detect. In the illustrated example, markers may be placed on shoulder point 20i, elbow point 20g, hand point 20f, and center point 20a. The marker system may allow for real time characterization of subject image 20, however, there are several disadvantages.
The primary disadvantage associated with markers is that markers must be placed on subject 4 of FIG. 1 in order to obtain the various characteristics in subject image 20. The physical placement of markers is often not possible in situations where subject characterization is desired of objects not suitable for marking. For example, subject characterization may be desired of an animal in the wild. In such a case, markers cannot be placed on the wild animal.
Another disadvantage of using markers is that subject 4 of FIG. 1 must wear the markers. There may be a number of situations where the subject is not capable of or does not desire to wear markers, therefore, making subject characterization difficult. Additionally, the marker system may not be capable of obtaining all of the desired characteristics of subject 4. For example, body box 20b, head box 20a, arm box 20d, shoulder box 20h and forearm box 20e are two-dimensional shapes which may require a large number of markers in order to adequately characterize through computer vision. Once again, increasing the number of markers only adds to the difficulty and burden of using the marker system.
Another method of subject characterization using computer vision is to perform pretraining. A computer vision system may be used to preliminarily take and analyze electronic images of subject 4 before the actual subject characterization takes place in a real life situation. In this method, the computer vision system may be able to more accurately characterize subject 4 in a real life situation. However, this assumes that subject 4 is available for pretraining. If subject 4 is a wild animal or another subject that is not available for pretraining, this method of subject characterization may not be feasible. Further, the change in conditions between the training setting and the real life setting may diminish any benefit derived from pretraining.
As discussed, prior art systems, including the marker system and the pretraining system, may not be feasible in real life situations. However, other systems that do not utilize markers or pretraining may not be capable of real time operations because of the extensive amount of computations needed to accomplish subject characterization. Typically, prior art systems use exhaustive algorithms to determine the desired characteristics of subject image 20.
The use of such exhaustive algorithms assumes that the prior art systems are even capable of accurately capturing a subject image. To further distinguish the different points or parts of subject image 20, prior art systems may repetitively compare the current subject image 20 with all the possible sizes, shapes and positions subject 4 may be in. Typically, this exhaustive approach is incapable of producing real-time results.
Thus, what is desired is a more efficient computer vision system suitable for subject characterization. By eliminating the exhaustive nature of the characterization process of prior art systems, real time subject characterizations may be achieved. The increased efficiency should also remove the need for using markers and extensive pretraining of the computer vision system due to their inherent impractical nature.